Error Design

We’re dissociated from the thing we’re controlling. We are not the machine, we control the machine, and in some ways, it controls us. We’re in a symbiotic relationship with such machines. But unlike the human adaptability (and emotional nature) we have, machines are better thought of as brute force mechanisms, instructions that execute with a kind of relentless confidence and impunity. They stumble through situations that are “unfamiliar,” that is, situations in which they encounter a state-of-the-world that isn’t anticipated by either the creators of that machine or its emergent programming (as in systems with self-directed reinforcement learning).

Such machines will inevitably encounter errors and exceptions.

Such machines will inevitably encounter errors and exceptions.